Replicating genotype–phenotype associations. Nature, 447: 655-660. Jun 1, 2007.

NCI-NHGRI Working Group on Replication in Association Studies The study of human genetics has recently undergone a dramatic transition with the com­ pletion of both the sequencing of the human genome and the mapping of human haplo­ types of the most common form of genetic variation, the single nucleotide polymorphism (SNP). In concert with this rapid expansion of detailed genomic information, cost-effective genotyping technologies have been developed that can assay hundreds of thousands of SNPs simultaneously. Together, these advances have allowed a systematic, even ‘agnostic’, approach to genome-wide interrogation, thereby relaxing the requirement for strong prior hypotheses. So far, comprehensive reviews of the pub­ lished literature, most of which reports work based on the candidate-gene approach, have demonstrated a plethora of questionable geno­ type–phenotype associations, replication of which has often failed in independent stud­ ies. As the transition to genome-wide asso­ ciation studies occurs, the challenge will be to separate true associations from the blizzard of false positives attained through attempts to rep­ licate positive findings in subsequent studies. The purpose of a replication study is to evalu­ ate a positive finding from a previous study, to provide credibility that the initial finding is valid. Replication is essential for establishing the credibility of a genotype–phenotype asso­ ciation, whether derived from candidate-gene or genome-wide association studies. However, there is a lack of agreement about what consti­ tutes a finding deserving of replication, what constitutes an adequate replication study and what constitutes a replication or refutation. Investigators and journal editors have offered guidelines for how to address this problem, but these initial efforts have been hampered by limited experience and conflicting empirical data. However, as evidence has accumulated, several instructive examples have emerged of genotype–phenotype associations being reproduced reliably in follow-up studies. These include peroxisome proliferator-activated receptor-γ (PPARG) and the transcription factor TCF7L2 (refs 14–19), related to diabetes; nucleotide-binding oligomerization domain containing 2 (NOD2) and Crohn’s disease; complement factor H (CFH) and age-related macular degeneration; and chromosome region 8q24 and prostate cancer risk. Many instances have arisen in which initial findings have not been reproduced in follow-up studies because of issues in either the initial study or the attempted replication. Small sample size is a frequent problem and can result in insufficient power to detect minor contri­ butions of one or more alleles. Similarly, small sample sizes can provide imprecise or incor­ rect estimates of the magnitude of the observed effects. Poor study design — particularly a lack of comparability between cases and controls — can increase the risk of biases because there can be heterogeneity in exposure to environ­ mental challenges and population stratifica­ tion. The latter arises when investigators fail to account for case–control differences in the genetic structure of the underlying population. Heterogeneity in classification of outcomes across studies can undermine the opportu­ nity to compare among them. Similarly, data ‘dredging’ can be a major problem, especially when criteria for defining phenotypes are altered to achieve statistical significance worthy of publication. Another challenge arises when follow-up studies analyse different variants. An example is the reported association between DTNBP1 and schizophrenia, initially identified in Irish pedigrees and ‘confirmed’ in independent European studies. Unfortunately, different risk alleles and haplotypes were reported in each study, making comparison difficult. Although it is plausible that more than one variant could contribute to schizophrenia risk at the DTNBP1 locus, it is difficult to draw this conclusion from the literature because followup studies have not consistently analysed the same markers or those in perfect linkage dis­ equilibrium (r = 1.0). Other recent examples for which initial reports of association have been inconsistently replicated include insu­ lin-induced gene 2 (INSIG2) and obesity, and cyclic-AMP-specific phosphodiesterase (PDE4D) and stroke. These have been accompanied by controversies about what actually constitutes replication. This paper presents the conclusions of a working group on the replication of geno­ type–phenotype associations — whether identified in genome-wide or candidate-gene studies — convened by the National Cancer Institute and the National Human Genome Research Institute. The group was composed of experts from diverse disciplines, including biostatistics, clinical medicine, epidemiology, genetics and scientific publishing. The purpose was to review the current state of the field and propose best practices for the design, conduct and publication of replication studies that aim to follow up notable findings, particularly in genome-wide association studies. The group addressed three topics. First, assessment of the validity and limitations of any single genetic association study. Second, criteria for establish­ ing replication in genetic association studies. Third, points to consider for publication of high-quality genotype–phenotype association reports (Box 1).